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1. Introduction 

As system designs become more complex, determining the best locations to add sensors and test points 
for the purpose of testing and monitoring these designs becomes more difficult. Not only must the 
designer take into consideration all real and potential faults of the system, he or she must also find 
efficient ways of detecting and isolating those faults. Because sensors and cabling take up valuable space 
and weight on a system, and given constraints on bandwidth and power, it is even more difficult to add 
sensors into these complex designs after the design has been completed. As a result, a number of software 
tools have been developed to assist the system designer in proper placement of these sensors during the 
system design phase of a project. One of the key functions provided by many of these software programs 
is a testability analysis of the system - essentially an evaluation of how “observable” the system behavior 
is using available tests. During the design phase, testability metrics can help guide the designer in 
improving the inherent testability of the design. This may include adding, removing, or modifying tests; 
breaking up feedback loops, or changing the system to reduce fault propagation. Given a set of test 
requirements, the analysis can also help to verify that the system will meet those requirements. Of course, 
a testability analysis requires that a software model of the physical system is available. For the analysis to 
be most effective in guiding system design, this model should ideally be constructed in parallel with these 
efforts. 

The purpose of this paper is to present the final testability results of the Advanced Diagnostic and 
Prognostic Testbed (ADAPT) after the system model was completed. The tool chosen to build the model 
and to perform the testability analysis with is the Testability Engineering and Maintenance System 
Designer (TEAMS-Designer). The TEAMS toolset is intended to be a solution to span all phases of the 
system, from design and development through health management and maintenance. TEAMS-Designer is 
the model-building and testability analysis software in that suite. 

2. The ADAPT Model 

The first step towards analyzing the testability of a system is to create the software model. The ADAPT 
model created in TEAMS-Designer is a multi-signal dependency model which captures the ADAPT 
system’s basic structure, interconnections, sensors, and failure modes. The model is then able to link a 
given failure mode with a test or multiple tests based on signals that are propagated through the structure 
of the model. In total, there are 712 failure sources in the ADAPT model and 281 tests corresponding to 
approximately 150 sensors on the system. Failure modes in the ADAPT model might include a relay 
being stuck open or closed, low batteiy impedance, or low voltage at a sensor. Signals (also known as 
functions) attached to these failure modes might include Power On, Relay Open, RelayClosed, 

Battery Output Current, and multiple others. Figure 1 shows a portion of the model representing one of 
two batteiy chargers, with typical fault modes and functions displayed. This component block connects 
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with other blocks via links, along which the functions propagate to capture the hierarchy and fault 
behavior of the whole system. 


Chaigef inn I Power_Generation_BCl[l] Power_Generation[2] ADAPT_Power_Subsysteiti 



Figure 1: Battery Charger component in TEAMS, with functions labeled 

Once the failure modes and signals are created, tests can then be attached to the model to test for these 
functions and implicate or clear a particular fault. A dependency between a test and a failure source is 
established if they both contain the same function and they are interconnected with links (with the test 
being downstream of the failure source). For example, a voltage sensor named El 165 on ADAPT is able 
to detect a low voltage, causing the test EI165VOLTAGEHI to fail. This test has one function 
associated with it: Power On. By tracing this function through the model from the test point back to the 
source, TEAMS is able to determine all possible locations in the model where this signal could have been 
interrupted. 

For ease of understanding, identical names were used for a test point and the test hosted by it. The names 
are also indicative of the category a specific test belongs to. For example, ESH141POSITIONCLOSED 
refers to a test that is performed using a contact position sensor in a switch, while EI125VOLTAGELO 
refers to a test that is performed on a voltage signal from sensor El 125. 

The initial ADAPT model was created by Qualtech Systems (QSI) as part of a phase II Small Business 
Innovation Research (SBIR) project (contract number NNA06AA51Z). At the top level, the model 
consists of five functional blocks representing Power Generation, Power Storage, Power Distribution, 
Loads, and Monitor Control. Each block groups components, switches, and sensors into a module. In 
Figure 2, the top level of the hierarchy is displayed, showing the functional blocks. These blocks are 
described in more detail below. 
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Figure 2: Top-level view of TEAMS ADAPT model 

Power Generation - The Power Generation module represents the function of supplying primary power 
to the testbed. Specifically, this consists of two battery chargers running on utility power and the 
associated switches and circuit breakers connecting these chargers to the Power Storage module. 

Power Storage - The Power Storage module in the ADAPT model contains three sets of two 12-volt DC 
lead acid batteries for delivering stored power to the loads. Other components in this block include 
temperature sensors attached to the batteries, panel meters for manual observation of the batteries, and 
circuit breakers and relays. 

Power Distribution - This module connects the Power Storage function to the Loads, allowing for 
multiple configurations between the three battery sets and two load banks. The switches inside this 
module also must enforce rules restricting a load bank from being powered by more than one battery set. 

Loads - This block contains two load banks, each bank with six (6) AC and two (2) DC outputs. In the 
present configuration, the load banks are designed to support primary and backup loads for a light bank of 
three bulbs, a single lamp, a water pump, and two fans. This functional block also contains temperature, 
light, flow, and rate sensors for monitoring of these loads, along with two AC inverters and the necessary 
switches and circuit breakers. 
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Monitor Control - The final block contains the control switches for changing between switch modes 
based on commands sent to the testbed. It also contains components to represent sensor outputs in the 
system which can be used for real-time monitoring of the system with tools such as TEAMS-RT. On the 
testbed, this block corresponds to the National Instruments backplanes and associated Lab VIEW 
software. 

Once the model was built, several types of labels were added to assist in testability analysis. Of these 
types, Ttechnology labels were used to logically divide up the model into two basic sections. Because the 
Monitor Control block does not represent electrical power system components on the testbed, it is 
described under a separate technology label (Monitor-Control) while the remaining four blocks are 
described under another label (AdaptPowerSystem). Technology labels are described in more detail under 
section 5 below. 

3. Failure Modes in the System 

In its present state, the ADAPT model has 712 failure sources identified. The AdaptPowerSystem portion 
of the model (all sections excluding the Monitor Control block) contains 489 failure sources. Most 
components have multiple failure states and fall into one of several categories. Relays, for instance, can 
fail in four basic ways: both the contact and the coil can independently fail open or closed. Table 1 lists 
the major components and their failure modes. 


Device Type 

Symbol in Model 

Faults 

Relay 

EY 

Coil Open 

Coil Closed 

Contact Open 

Contact Closed 

Relay Contact Position Sensor 

ESH 

Sensor Open 

Sensor Closed 

Light Transducer 

LT 

Low Signal (Failed Low) 

Circuit Breaker 

CC/BC 

Stuck Open 

Stuck Closed 

Position Closed 

Position Open 

Circuit Breaker Position Sensor 

ISH 

Sensor Open 

Sensor Closed 

Voltage Indicator 

El 

Stuck High 

Stuck Low 

Charge Controller 

CHG_CTLR 

Open 

Short 

Impedance Low 

Battery Charger 

Battery Charger 

Short 

Open 

BC Overcharge 

BC Impedance Low 

Current Transducer 

IT 

Stuck Open 

Stuck Closed 

Battery 

Battery 

Impedance Low 

Voltage Low 

Overheated 
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Panel Meter 

PNL_MTR 

Meter Indicator Fault 

Temperature Sensor 

TE 

Output High 

Frequency Sensor 

ST 

Sensor Open 

Sensor Closed 

Flow Sensor 

FT 

Sensor Open 

Sensor Closed 

Loads 

Load 

Open 

Short 

Impedance Fault 

Inverter 

Inverter 

Short 

Frequency Fault 

Low Output 


Table 1: Failure sources in TEAMS ADAPT model, listed by device type 

4. Testability Analysis 

Once the process of adding tests begins, the testability analysis is a good way to determine the best 
locations for adding additional test points. One reason for using a testability analysis to guide sensor 
placement is to trade off the cost of adding additional sensors against the benefit gained by increasing the 
information available for monitoring system behavior. The risks of making uninformed decisions 
regarding sensors and sensor placement are summarized nicely here: 

There is no straightforward relation betw’een the number of sensors and diagnosability of the 
systems; increasing the number of sensors alone does not guarantee a higher level of 
diagnosability. The relevance of information provided by an additional sensor and its 
correlation with information provided by other sensors must also be taken into account. Besides 
the issue of diagnosability >, we also consider economics issues. We must provide a sensor system 
that achieves a desired degree of diagnosability > at the lowest possible cost. [3 ] 

Inherent tradeoffs involved in adding additional sensors to the system include sensor cost, sensor weight, 
bandwidth limitations for transmitting telemetry and other considerations. For flight applications, and 
space flight in particular, weight plays a crucial role in determining what can be added to the system. If 
current sensors, for instance, generally weigh more than voltage sensors, then voltage sensors might be a 
better choice if the gain/loss in fault detection capability between them. The designers must take into 
account these factors while also ensuring a certain amount of redundancy to allow for potential sensor 
failures. 

One thing to note when running a testability analysis is that TEAMS-Designer makes a single-fault 
assumption when building its diagnostic tree. This means that it will not consider the possibility of 
multiple faults combining to give the same signature as a single fault. If this were not the case, the tree 
would grow to an overwhelmingly large size to include double, triple, and even larger fault combinations. 
A single-fault assumption also significantly helps to reduce the size of ambiguity groups. 
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5. Testability Options 


There are several options available to the user when running a testability analysis in TEAMS-Designer. 
Many of these options are tied to how the user chooses to model their system. More granularity may be 
achieved by using options such as technologies and test labels to limit an analysis to specific parts of the 
model. This section will describe some of the options available to do this. For reference, Figure 3 displays 
the menu where many of these selections are specified. 



Figure 3: TEAMS Testability Analysis Options pop-up menu 

Analysis For - The simplest way to limit a testability analysis, this option allows the user to choose a 
specific module/component for analysis. This can either be a top-level component containing other 
components or a lower-level failure mode. If something other than the top-level “system” component is 
chosen, then either global or local analysis should also be selected. Global analysis takes into account 
failures external to that component but assigns their failure rate to zero. Local analysis only looks at local 
components and local failures inside the module specified. The result of local analysis is generally a 
smaller ambiguity group set with a smaller number of components in each group, giving a higher isolation 
rate. This setting might be useful for more effective fault isolation if there are a large number of faults 
propagating from one module to another. 
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Technologies - Another way to restrict analysis to a particular set of modules is to use Technologies. 
When building the model, a technology label can be applied to any component and a component may fall 
under multiple technologies. During testability analysis, multiple technologies may be selected. If 
technologies are not used it is assumed all components (and technologies) are to be included in the 
analysis. The primary reason for using a technology label during testability analysis is to exclude any 
failure sources outside of those technologies. The ADAPT model is divided into two major technology 
labels: AdaptPowerSystem and Monitor-Control (as previously discussed in section 2). 

Test Labels - It is possible to not only limit the analysis at a component level, but also at the level of 
tests. Any test point may belong to a particular class grouped by a common label. These test labels are 
user defined, and when running a testability analysis the user can choose any number of test labels to 
isolate to. Limiting an analysis using test labels does not decrease the number of failure sources in the 
problem set unless combined with technology labels; instead, it limits the number of tests used in fault 
detection and isolation. In the ADAPT model, test labels distinguish between different types of sensors, 
such as voltage, current, or temperature sensors. 

System Modes - In TEAMS, a system mode is a collection of switch modes within the model. Because 
many of the test points should only be active in a given system configuration (i.e., it is unnecessary to test 
for high voltage at the loads when the loads are not connected to the battery) it is a requirement that these 
configurations be separately defined but also included as part of a testability analysis in order to obtain a 
complete view of the system. 

Test Method - There are three test methods defined by default in TEAMS: Manual, Automatic, and On- 
Board (BIT) Test. Depending on the kind of tests involved, there might be greater or lesser coverage in 
the testability analysis. A manual test could include the use of a voltmeter or visual observation, 
potentially useful information in order to fully isolate a fault within a given candidate list. The ADAPT 
model contains 242 automatic tests, 39 manual tests, and 0 on-board tests. Most of the manual tests also 
correspond to the “Crew Observed” test label and include such questions as, “Is Light A on?” 

Finally, TEAMS allows the user to isolate to either components or failure modes (the lowest-level 
component). It is also possible to isolate to a particular level of hierarchy within the model. For the 
purpose of this paper, we are primarily interested in isolating to failure modes. 

6. Testability Output 

After performing a testability analysis in TEAMS-Designer, the user is presented with a number of 
testability figures of merit (TFOMs) along with other information to help gauge the observability of the 
model under test. An example of the testability report is shown in Figure 4. 
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Figure 4: TEAMS Testability Analysis Report example output 

Fault Detection vs. Fault Isolation - The two principal metrics for any testability analysis are fault 
detection and fault isolation. Respectively, these metrics measure the ability of the system to detect the 
presence of a fault that has occurred, and to identify that particular fault and distinguish it without 
ambiguity from other faults based on available data. If the fault detection of a system is very high, then it 
can be said to have high fault observability. The TEAMS-Designer tool reports these numbers as 
Percentage Fault Detection and Percentage Fault Isolation. 

Ambiguity Group Size - Another related metric to provide additional insight into the system is 
ambiguity group size. The ambiguity group size indicates the level of fault isolation available in the 
model. If a given fault cannot be isolated to a single component, it will have an ambiguity group size of 
greater than one. If there are many large ambiguity groups, it may be necessary to add additional tests to 
the system to break these groups up. At the same time, it may not be necessary to isolate all faults to 
individual components if they exist within a single Line Replaceable Unit (LRU). 

Number of tests - As the name indicates, this is the number of tests being used in the current testability 
analysis. This number includes test points throughout the entire ADAPT model, but may be limited by 
choosing a subset of test labels. 

Tests not used - These are tests that were not needed for the section of the model being tested. A test will 
not be used for any of the following three reasons: (a) it has no coverage, meaning it does not detect 
anything - this could be the case if using switch/system modes and the path to the test point is broken, 
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(b) it has the same test/dependency signature as another cheaper test, or (c) the same faults can be isolated 
using a combination of other tests. 

Number of failure sources - The number of failure modes in the model or section of the model under 
test. 

7. Test Results 

A testability analysis was run on the model using a variety of test labels and using both automatic and 
manual test methods. The goal of this analysis was to identify the most efficient of the sensor groups and 
to potentially identify ways of increasing efficiency in fault detection or fault isolation. For the examples 
presented in this paper, all system modes were used and therefore the maximum number of tests in the 
system were made available for fault detection and isolation. 

In the first round of tests, the most useful comparison deals with using different types of sensors to detect 
the entire fault set. The analysis was isolated to the AdaptPowerSystem technology group, which 
represents the electrical power system components of ADAPT. A testability analysis was performed for 
this technology group using each category of test label, each label representing a different sensor or input 
type. This information is useful in measuring how a given sensor type performs in detecting faults in the 
model exclusive of any other sensors. 

To start with, the testability analysis gives some basic, useful information about the system. This 
information includes how many modules, failure sources, tests, and switches are present in the section of 
the model under analysis. Table 2 displays these attributes for components in the AdaptPowerSystem 
technology group. 


Number of failure sources 

488 

Number of switches 

616 

Number of dependencies 

3948 

Number of modules at level 1 

5 

Number of modules at level 2 

17 

Number of modules at level 3 

161 

Number of modules at level 4 

502 

Number of modules at level 5 

374 


Table 2: AdaptPowerSystem technology label attributes 

The number of modules at each level begins with the topmost level of the hierarchy, level one, and moves 
down toward the individual failure modes defined in the model. The failure sources are mostly located at 
the bottom two levels, but they can also be found throughout the model. 
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In the table below we break down the types of sensors by test label and provide the results of the analysis. 


Test Label 

All 

Crew 

Observed 

Panel 

Meters 

Current 

Sensors 

Voltage 

Sensors 

Frequency 

Sensors 

Flow 

Sensors 

Temp. 

Sensors 

CB 

Position 

Sensors 

Relay 

Position 

Sensors 

Light 

Meters 

Percentage 

Fault 

Detection 

98.36% 

29.51% 

15.98% 

31.76% 

39.14% 

14.34% 

13.11% 

18.44% 

12.30% 

32.99% 

13.11% 

Percentage 

Fault 

Isolation 

73.03% 

3.25% 

0.61% 

6.90% 

7.10% 

0.00% 

0.20% 

1.22% 

0.81% 

0.00% 

0.00% 

Percentage 
Retest OK's 

16.43% 

87.01% 

92.69% 

84.92% 

82.45% 

95.89% 

96.70% 

92.69% 

93.71% 

84.28% 

97.11% 

Ambiguity 
Group Size 

1.59 

245.06 

346.40 

229.93 

183.80 

360.14 

370.77 

326.81 

377.04 

221.59 

370.98 

Number of 
tests 

281 

32 

7 

42 

64 

6 

4 

20 

28 

76 

2 

Number of 
tests not used 

45 

0 

0 

7 

18 

0 

0 

5 

0 

0 

0 

Number of 
nodes in tree 

821 

127 

69 

147 

171 

39 

31 

71 

61 

153 

27 

Efficiency of 
Test Sequence 

6.82% 

8.80% 

9.65% 

6.86% 

6.90% 

14.12% 

15.56% 

10.14% 

6.05% 

4.76% 

18.93% 


Table 3: AdaptPowerSystem testability analysis results using a single test label 

The number of tests for individual test labels adds up to the total for all test labels, so no unlabeled tests 
have been excluded from the model. However, the number of unused tests for each test label does not add 
up to the total unused tests, indicating by combining tests from various types of sensors, some of the 
testing that would otherwise be necessary can be eliminated. This evidence highlights something that is 
often overlooked in sensor design and placement, and that is the interplay between sensors. 


Judging from the numbers shown in Table 3, it becomes clear that the voltage sensors have higher rates of 
both fault detection and fault isolation than other sensor groups when used in isolation. Current sensors 
and relay position sensors also were represented fairly well, although relay sensors were not as effective 
at fault isolation. Unfortunately, a problem with basing conclusions solely on the table above is that the 
percentage of fault detection for a particular type of test does not take into account the other tests that 
might also be able to detect those same faults. In the next dataset the same analysis was run, but rather 
than isolate each test label, the full set of sensors was used and a single test label was removed from the 
set. This test makes the usefulness of a particular type of sensor clearer. From these numbers it can be 
determined just how much “testability” of the system is lost if one type of test is completely removed. 
Using 98.36% and 73.03% to represent baseline testability using all sensors, we calculate the percentage 
lost by removing each sensor group in the final two rows. The results of this test can be seen in Table 4. 
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Test Label 

All 

No 

Crew 

Observed 

No 

Panel 

Meters 

No 

Current 

Sensors 

No 

Voltage 

Sensors 

No 

Frequency 

Sensors 

No 

Flow 

Sensors 

No 

Temp. 

Sensors 

No 

CB 

Position 

Sensors 

No 

Relay 

Position 

Sensors 

No 

Light 

Meters 

Percentage 

Fault 

Detection 

98.36% 

92.42% 

96.31% 

88.52% 

89.34% 

96.72% 

97.54% 

93.85% 

93.44% 

84.22% 

97.95% 

Percentage 

Fault 

Isolation 

73.03% 

59.44% 

68.16% 

62.28% 

57.61% 

72.22% 

72.22% 

69.38% 

61.06% 

37.53% 

72.63% 

Number of 
tests 

281 

249 

274 

239 

217 

275 

277 

261 

253 

205 

279 

Number of 
tests not used 

45 

45 

43 

33 

14 

45 

45 

40 

41 

41 

45 

Percentage Fault 
Detection Lost 

N/A 

5.94% 

2.05% 

9.84% 

9.02% 

1.64% 

0.82% 

4.51% 

4.92% 

14.14% 

0.41% 

Percentage Fault 
Isolation Lost 

N/A 

13.59% 

4.87% 

10.75% 

15.42% 

0.81% 

0.81% 

3.65% 

11.97% 

35.50% 

0.40% 


Table 4: AdaptPowerSystem testability analysis results using all tests, except test label indicated 

From these data it can be seen that in terms of fault detection, the most comprehensive sensor groups 
seem to be the relay position sensors, current sensors, and voltage sensors. It should be noted that these 
numbers are skewed towards those labels with a greater number of sensors compared to other sensor 
groups. Unfortunately a one-to-one comparison between test labels is difficult to quantify because some 
of the sensors are specific to certain parts of the model and to certain signals in particular. The flow 
sensors and light meters, for instance, are only applicable at the load level and it would be useless to add 
additional sensors of this type to the testbed in order to determine the system health at another location. 


As mentioned earlier, the addition of sensor combinations shows increasing gains in the number of tests 
not used. Looking at these numbers, it would appear that voltage sensors contribute the most to reducing 
the number of tests needed in a diagnosis of the system. This number is misleading, however, because it 
includes voltage sensor tests that would otherwise not have been added. Several of the other test labels 
offer higher gains here, after filtering out test points of that test label type. CB position sensors reduce the 
number of tests needed by four (4), relay sensors reduce the number of tests by four (4), current sensors 
reduce the tests by three (3), and panel meters reduce the tests by two (2) in conjunction with other test 
groups and exclusive of their respective sensor type. Unfortunately these gains are only realized when 
multiple system modes in the model are used. As the number of system modes decrease, the number of 
accessible tests becomes fewer, eliminating these combinations. It would also be incorrect to assume that 
any sensors can be eliminated from the system without a loss in testability. The majority of sensors are 
represented in the model by at least two different test points: a high and a low test, or an open and closed 
test for relay sensors and circuit breakers. Some voltage and current sensors are represented by three test 
points, one for each battery configuration. Of the redundant tests, none include all test points for a given 
sensor. 
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8. Test Results by Module 


In the following series of tables (Tables 5-8) the testability analysis is divided among the four main top- 
level components. By isolating these functional blocks, one can see which areas of the testbed are more or 
less observable as well as which sensor groups are most effective at fault detection and isolation for each 
block. The “local” setting was used when running each testability analysis on the component group, so 
only failure sources local to the component are considered. All tests in the model of the given “Test 
Label” type are included. 


Test Label 

All 

Crew 

Observed 

Panel 

Meters 

Current 

Sensors 

Voltage 

Sensors 

Frequency 

Sensors 

Flow 

Sensors 

Temp. 

Sensors 

CB 

Position 

Sensors 

Relay 

Position 

Sensors 

Light 

Meters 

Percentage 

Fault 

Detection 

100.00% 

12.90% 

22.58% 

58.06% 

38.71% 

12.90% 

12.90% 

25.81% 

12.90% 

16.13% 

12.90% 

Percentage 

Fault 

Isolation 

61.70% 

0.00% 

3.19% 

3.19% 

15.96% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Ambiguity 
Group Size 

1.57 

72.04 

57.1 

18.92 

36.48 

72.04 

72.04 

52.88 

71.78 

66.80 

72.04 

Number of 
failure sources 

93 

93 

93 

93 

93 

93 

93 

93 

93 

93 

93 

Number of 
tests 

281 

32 

7 

42 

64 

6 

4 

20 

28 

76 

2 

Number of tests 230 
not used 

31 

3 

26 

45 

5 

3 

13 

22 

70 

1 


Table 5: ADAPT Power Storage component, testability analysis results by test label 

The Power Storage module is one of the most diversified of the top-level components in terms of sensors. 
It contains current and voltage sensors, CB position sensors, relay sensors, temperature sensors, and panel 
meters. The module is shown to have 100% fault detection, but only 61.70% fault isolation. Looking 
more closely at the components that make up this module, part of the reason for the low isolation rate is 
due to the battery. Rather than using a true 24-volt battery, the ADAPT testbed uses two 12-volt batteries 
connected in series. Because the current sensors downstream of the battery cannot differentiate between 
an increase in the current of one battery in the set versus another - possibly indicating a low-impedance 
fault - an ambiguity group is created for each battery set. A similar issue exists for the low-voltage 
condition between the batteries. If each set of batteries were replaced with a single 24-volt battery, using 
only existing sensors the Percentage Fault Isolation for Power Storage would increase to 74.33% 
(+12.63%) and overall Percentage Fault Isolation for the system increases to 75.50% (+2.47%). 
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Test Label 

All 

Crew 

Observed 

Panel 

Meters 

Current 

Sensors 

Voltage 

Sensors 

Frequency 

Sensors 

Flow 

Sensors 

Temp. 

Sensors 

CB 

Position 

Sensors 

Relay 

Position 

Sensors 

Light 

Meters 

Percentage 

Fault 

Detection 

97.32% 

42.86% 

10.71% 

25.00% 

33.93% 

17.86% 

15.18% 

16.07% 

11.61% 

44.64% 

14.29% 

Percentage 

Fault 

Isolation 

87.51% 

0.00% 

0.00% 

5.33% 

2.67% 

0.00% 

0.00% 

0.00% 

0.89% 

0.00% 

0.00% 

Ambiguity 
Group Size 

1.31 

75.33 

179.95 

127.83 

99.94 

152.93 

163.31 

159.67 

176.35 

70.71 

166.69 

Number of 
failure sources 

224 

224 

224 

224 

224 

224 

224 

224 

224 

224 

224 

Number of 
tests 

281 

32 

7 

42 

64 

6 

4 

20 

28 

76 

2 

Number of 
tests 
not used 

152 

0 

3 

27 

44 

0 

1 

16 

16 

28 

0 


Table 6: ADAPT Power Distribution component, testability analysis results by test label 

The Power Distribution component is also fairly diversified in its sensors, represented here in Table 6. 
This module has the greatest numbers of failure sources by far, more than double that of the next highest 
module. This helps to explain why the Percentage Fault Isolation for individual test labels is extremely 
low and the Ambiguity Group Size for each label is very high. Obviously it will be more difficult to 
detect and isolate faults to the same percentage as in a module with fewer fault modes. Surprisingly, 
despite the low individual numbers, when all tests are combined it still reaches above 80% fault isolation. 
The Crew Observed tests are surprisingly effective for fault detection here because of this module’s 
strong ties to the Loads module. This module has the largest number of relays in the system, explaining 
the high fault detection gained from using relay position sensors. 


Test Label 

All 

Crew 

Observed 

Panel 

Meters 

Current 

Sensors 

Voltage 

Sensors 

Frequency 

Sensors 

Flow 

Sensors 

Temp. 

Sensors 

CB 

Position 

Sensors 

Relay 

Position 

Sensors 

Light 

Meters 

Percentage 

Fault 

Detection 

98.02% 

0.00% 

32.67% 

28.71% 

62.38% 

0.00% 

0.00% 

1.98% 

21.78% 

45.54% 

0.00% 

Percentage 

Fault 

Isolation 

64.71% 

0.00% 

0.00% 

2.94% 

0.98% 

0.00% 

0.00% 

1.96% 

1.96% 

0.00% 

0.00% 

Ambiguity 
Group Size 

1.71 

102 

47.83 

53.47 

17.71 

102 

102 

98.06 

63.28 

31.71 

102 

Number of 
failure sources 

101 

101 

101 

101 

101 

101 

101 

101 

101 

101 

101 

Number of 
tests 

281 

32 

7 

42 

64 

6 

4 

20 

28 

76 

2 

Number of 
tests 
not used 

237 

32 

4 

34 

49 

6 

4 

18 

18 

54 

2 


Table 7: ADAPT Power Generation component, testability analysis results by test label 
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The Power Generation block includes the complex switching that enables either of the two battery 
chargers to connect with any of the three battery components. As seen in Table 7 above, several of the 
sensor types are unable to detect any fault modes in this block. As mentioned earlier, the reason for this 
lies in the fact that many of the sensors are load specific and do not measure any components in the Power 
Generation side of the model. The Ambiguity Group Size of 102 for these tests is one more than the 
number of failure sources because the Ambiguity Group includes all failure sources and one additional 
state. In TEAMS, this state corresponds to “All Systems Go” and denotes the condition wherein there 
exist no faults in the system. 


In the Power Generation block analysis, the Percentage Fault Isolation is one of the lowest among any of 
the other top-level component blocks. Closer examination of the ambiguity groups reveals that one set of 
switches that connects the chargers to the batteries has no test that can isolate between the three failed 
closed states of each switch. The ambiguity group for relay EY206 for instance, includes the following 
failure modes: ESH206A_Sensor_Closed, EY206_Coil_closed, and EY206_Contact_Closed. A similar 
situation exists for relay EY306. The Power Generation block contains no voltage or current sensors, and 
the testability numbers these tests provide are from sensors downstream of the battery chargers. An 
additional voltage sensor within this block would help to increase the fault isolation ability of the system 
by breaking up these ambiguity groups. 


Other ambiguity groups that could similarly be broken up by the addition of voltage sensors exist in the 
Contact Closed fault mode for the set of switches EY1 15, EY215, EY315, EY1 16, EY216, EY316, 
EY1 17, EY217, and EY317. There is currently no downstream test to distinguish between any three of 
the switches connected to one battery bus. A voltage sensor would need to be added immediately after 
each of these relays in order to increase isolation rate. 


Test Label 

All 

Crew 

Observed 

Panel 

Meters 

Current 

Sensors 

Voltage 

Sensors 

Frequency 

Sensors 

Flow 

Sensors 

Temp. 

Sensors 

CB 

Position 

Sensors 

Relay 

Position 

Sensors 

Light 

Meters 

Percentage 

Fault 

Detection 

100.00% 

51.43% 

0.00% 

22.86% 

22.86% 

25.71% 

25.71% 

40.00% 

0.00% 

0.00% 

28.57% 

Percentage 

Fault 

Isolation 

100.00% 

42.25% 

0.00% 

22.54% 

22.54% 

2.82% 

2.82% 

5.63% 

0.00% 

0.00% 

0.00% 

Ambiguity 
Group Size 

1 

17.86 

71 

42.78 

42.78 

40.5 

40.5 

26.93 

71 

71 

37.99 

Number of 
failure sources 

70 

70 

70 

70 

70 

70 

70 

70 

70 

70 

70 

Number of 
tests 

281 

32 

7 

42 

64 

6 

4 

20 

28 

76 

2 

Number of tests 
not used 

239 

16 

7 

38 

60 

2 

0 

12 

28 

76 

0 


Table 8: ADAPT Loads component, testability analysis results by test label 

Looking at testability metrics for the Loads component in Table 8, the main point of note is that 100% of 
the faults at the load level are both detectable and isolatable while using only 42 of the available tests. 
There are two main reasons for this. First of all, the physical loads are heavily sensored with an array of 
different sensor types. The light bank attached to AC Load Al, for example, uses both a light sensor and 
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individual temperature sensors for each bulb. A subset of these sensors can help distinguish between a 
complete failure of the light bank and a partial failure involving only one or two of the light bulbs. 

Second, there are a number of crew-observed tests that can tell whether a load is on or off through visual 
observation. The user can often get a good indication of how the system is running by monitoring the 
loads, so this is an ideal place for such tests. The effect of removing these crew observations as test points 
is discussed briefly below. A final reason for the high testability percentages is the relatively low number 
of failure sources compared to the other top-level modules. 

The Loads block contains no relays or circuit breakers; all switching is performed at the Power 
Distribution block. Another interesting fact that was discovered through the use of the testability analysis 
is that the load sensors (flow, temperature, frequency, and light sensors) are good at fault detection, but 
they provide only a small percentage of the fault isolation. The crew-observed tests provided the most in 
terms of both fault detection and isolation. 

Unfortunately, the fact that manual crew-observed tests play such an important role in Loads testability 
means that most automated diagnostic reasoners would be unable to reach such a high percentage of 
detection or isolation without at least some input from the user. Not all reasoners allow for this input, and 
on flight hardware some failure situations must be resolved more quickly than is possible with human 
feedback. As expected, there was a fairly substantial drop in testability when the same analysis was run 
without the benefit of crew-observed tests - total Percentage Fault Detection fell to 88.57% (-1 1.43%) 
while total Percentage Fault Isolation fell to 70.42% (-29.58%) for the Loads component. 

9. Conclusion 

Several result sets from the ADAPT TEAMS testability analysis are presented here, covering the testbed 
as a whole and also broken down by functional blocks. From these results, the primaiy conclusions from 
the testability analysis study can be summarized as follows: First, the voltage sensors, current sensors, 
and relay sensors are the most effective in terms of fault isolation and fault detection, though they are also 
the most numerous among the sensor groups. Second, by combining tests from various types of sensors, 
some of the testing that would otherwise be necessary can be eliminated. The most effective sensor 
groups for eliminating extra tests are circuit breaker position sensors, relay sensors, and current sensors. 
Unfortunately, these gains are diminished when using a subset of available system modes. It was also 
found that none of the existing sensors could be removed without at least some loss in testability. Third, a 
number of locations in the testbed were identified where the fault detection or isolation could be increased 
by making system changes or by adding additional sensors. Switching the power source to a single 24- 
volt battery or adding additional voltage sensors to the power generation section of the testbed are two of 
these examples. Finally, the crew-observed tests play a larger role in testability than originally predicted, 
especially when diagnosing faults in the Loads section of the ADAPT testbed. This would seem to stress 
the importance of human observations in conjunction with automated diagnosis when doing fault 
detection and isolation. 
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